Developing a complex intervention to improve prescribing safety in primary care:mixed methods feasibility and optimisation pilot study

Objectives (A) To measure the extent to which different candidate outcome measures identified high-risk prescribing that is potentially changeable by the data-driven quality improvement in primary care (DQIP) intervention.(B) To explore the value of reviewing identified high-risk prescribing to clinicians.(C) To optimise the components of the DQIP intervention.  Design Mixed method study.  Setting General practices in two Scottish Health boards.  Participants 4 purposively sampled general practices of varying size and socioeconomic deprivation.  Outcome measures Prescribing measures targeting (1) high-risk use of the non-steroidal anti-inflammatory drugs (NSAIDs) and antiplatelets; (2) ‘Asthma control’ and (3) ‘Antithrombotics in atrial fibrillation (AF)’.  Intervention The prescribing measures were used to identify patients for review by general practices. The ability of the measures to identify potentially changeable high-risk prescribing was measured as the proportion of patients reviewed where practices identified a need for action. Field notes were recorded from meetings between researchers and staff and key staff participated in semistructured interviews exploring their experience of the piloted intervention processes.  Results Practices identified a need for action in 68%, 25% and 18% of patients reviewed for prescribing measures (1), (2) and (3), respectively. General practitioners valued being prompted to review patients, and perceived that (1) ‘NSAID and antiplatelet’ and (2) ‘antithrombotics in AF’ were the most important to act on. Barriers to initial and ongoing engagement and to sustaining improvements in prescribing were identified.  Conclusions ‘NSAIDs and antiplatelets’ measures were selected as the most suitable outcome measures for the DQIP trial, based on evidence of this prescribing being more easily changeable. In response to the barriers identified, the intervention was designed to include a financial incentive, additional ongoing feedback on progress and reprompting review of patients, whose high-risk prescribing was restarted after a decision to stop.  Trial registration number Clinicaltrials.govNCT01425502

Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial:Quantitative examination of variation between practices in recruitment, implementation and effectiveness

Objectives: - The cluster randomised trial of the Data-driven Quality Improvement in Primary Care (DQIP) intervention showed that education, informatics and financial incentives for general medical practices to review patients with ongoing high-risk prescribing of non-steroidal anti-inflammatory drugs and antiplatelets reduced the primary end point of high-risk prescribing by 37%, where both ongoing and new high-risk prescribing were significantly reduced. This quantitative process evaluation examined practice factors associated with (1) participation in the DQIP trial, (2) review activity (extent and nature of documented reviews) and (3) practice level effectiveness (relative reductions in the primary end point). Setting/participants: - Invited practices recruited (n=33) and not recruited (n=32) to the DQIP trial in Scotland, UK. Outcome measures: - (1) Characteristics of recruited versus non-recruited practices. Associations of (2) practice characteristics and 'adoption? (self-reported implementation work done by practices) with documented review activity and (3) of practice characteristics, DQIP adoption and review activity with effectiveness. Results: - (1) Recruited practices had lower performance in the quality and outcomes framework than those declining participation. (2) Not being an approved general practitioner training practice and higher self-reported adoption were significantly associated with higher review activity. (3) Effectiveness ranged from a relative increase in high-risk prescribing of 24.1% to a relative reduction of 77.2%. High-risk prescribing and DQIP adoption (but not documented review activity) were significantly associated with greater effectiveness in the final multivariate model, explaining 64.0% of variation in effectiveness. Conclusions: - Intervention implementation and effectiveness of the DQIP intervention varied substantially between practices. Although the DQIP intervention primarily targeted review of ongoing high-risk prescribing, the finding that self-reported DQIP adoption was a stronger predictor of effectiveness than documented review activity supports that reducing initiation and/or re-initiation of high-risk prescribing is key to its effectiveness

A balanced approach to identifying, prioritising and evaluating all potential consequences of quality improvement:modified Delphi study

Objectives Healthcare is a complex system, so quality improvement will commonly lead to unintended consequences which are rarely evaluated. In previous qualitative work, we proposed a framework for considering the range of these potential consequences, in terms of their desirability and the extent to which they were predictable or expected during planning. This paper elaborates on the previous findings, using consensus methods to examine what consequences should be identified, why and how to prioritise, evaluate and interpret all identified consequences, and what stakeholders should be involved throughout this process. Design Two-round modified Delphi consensus study. Setting and participants Both rounds were completed by 60 panellists from an academic, clinical or management background and experience in designing, implementing or evaluating quality improvement programmes. Results Panellists agreed that trade-offs (expected undesirable consequences) and unpleasant surprises (unexpected undesirable consequences) should be actively considered. Measurement of harmful consequences for patients, and those with high workload or financial impact was prioritised, and their evaluation could also involve the use of qualitative methods. Clinical teams were agreed as important to involve at all stages, from identifying potential consequences, prioritising which of those to systematically evaluate, undertaking appropriate evaluation and interpreting the findings. Patients were necessary in identifying consequences, managers in identifying and prioritising, and improvement advisors in interpreting the data. Conclusion There was consensus that a balanced approach to considering all the consequences of improvement can be achieved by carefully considering predictable trade-offs from the outset and deliberately pausing after implementation to identify any unexpected surprises and make an informed decision as to whether quantitative or qualitative evaluation is needed and feasible. Stakeholders' roles in in the process of identifying, prioritising, evaluating and interpreting potential consequences should be explicitly addressed within planning and revisited during and after implementation

Balancing measures or a balanced accounting of improvement impact:a qualitative analysis of individual and focus group interviews with improvement experts in Scotland

Background As quality improvement (QI) programmes have become progressively larger scale, the risks of implementation having unintended consequences are increasingly recognised. More routine use of balancing measures to monitor unintended consequences has been proposed to evaluate overall effectiveness, but in practice published improvement interventions hardly ever report identification or measurement of consequences other than intended goals of improvement. Methods We conducted 15 semistructured interviews and two focus groups with 24 improvement experts to explore the current understanding of balancing measures in QI and inform a more balanced accounting of the overall impact of improvement interventions. Data were analysed iteratively using the framework approach. Results Participants described the consequences of improvement in terms of desirability/undesirability and the extent to which they were expected/unexpected when planning improvement. Four types of consequences were defined: expected desirable consequences (goals); expected undesirable consequences (trade-offs); unexpected undesirable consequences (unpleasant surprises); and unexpected desirable consequences (pleasant surprises). Unexpected consequences were considered important but rarely measured in existing programmes, and an improvement pause to take stock after implementation would allow these to be more actively identified and managed. A balanced accounting of all consequences of improvement interventions can facilitate staff engagement and reduce resistance to change, but has to be offset against the cost of additional data collection. Conclusion Improvement measurement is usually focused on measuring intended goals, with minimal use of balancing measures which when used, typically monitor trade-offs expected before implementation. This paper proposes that improvers and leaders should seek a balanced accounting of all consequences of improvement across the life of an improvement programme, including deliberately pausing after implementation to identify and quantitatively or qualitatively evaluate any pleasant or unpleasant surprises

Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: Case study evaluation of adoption and maintenance of a complex intervention to reduce high-risk primary care prescribing

Objective To explore how different practices responded to the Data-driven Quality Improvement in Primary Care (DQIP) intervention in terms of their adoption of the work, reorganisation to deliver the intended change in care to patients, and whether implementation was sustained over time.  Design Mixed-methods parallel process evaluation of a cluster trial, reporting the comparative case study of purposively selected practices.  Setting Ten (30%) primary care practices participating in the trial from Scotland, UK.  Results Four practices were sampled because they had large rapid reductions in targeted prescribing. They all had internal agreement that the topic mattered, made early plans to implement including assigning responsibility for work and regularly evaluated progress. However, how they internally organised the work varied. Six practices were sampled because they had initial implementation failure. Implementation failure occurred at different stages depending on practice context, including internal disagreement about whether the work was worthwhile, and intention but lack of capacity to implement or sustain implementation due to unfilled posts or sickness. Practice context was not fixed, and most practices with initial failed implementation adapted to deliver at least some elements. All interviewed participants valued the intervention because it was an innovative way to address on an important aspect of safety (although one of the non-interviewed general practitioners in one practice disagreed with this). Participants felt that reviewing existing prescribing did influence their future initiation of targeted drugs, but raised concerns about sustainability.  Conclusions Variation in implementation and effectiveness was associated with differences in how practices valued, engaged with and sustained the work required. Initial implementation failure varied with practice context, but was not static, with most practices at least partially implementing by the end of the trial. Practices organised their delivery of changed care to patients in ways which suited their context, emphasising the importance of flexibility in any future widespread implementation

Parallel process evaluation using a proposed framework for the design and reporting of process evaluations for cluster-randomised trials of complex interventions.

Process evaluations are recommended to open the 'black box' of complex interventions evaluated in trials, but there is limited guidance to help with design, with most guidance focused on the use of qualitative methods rather than processes to evaluate. We have developed a framework that identifies a range of candidate processes that an evaluation could examine to understand clusterrandomised intervention implementation and maintenance. These include processes involving clusters (recruitment, delivery of the intervention to the cluster, adoption by the cluster), involving the individuals on whom outcomes are measured (reach, delivery of the intervention to the individual, response of the individual), maintenance, and unintended consequences. Researchers will often have to choose what to focus on because of resource constraints, the framework aims to help researchers make explicit their choices of research questions and methods. We will illustrate application of the framework to the design and conduct of a parallel process evaluation of the Data-driven Quality Improvement in Primary Care trial, which evaluates a complex intervention to improve prescribing safety in 40 general practices. This mixed method process evaluation includes a dominant case study approach in a purposeful sample of practices and hypothesis?testing quantitative analysis of data from all practices examining how key processes are associated with variation in outcome between practices. We will present analysis of qualitative data from the 10 case study practices participating (6 reducing and 4 not reducing targeted prescribing), and focus groups with patients who have had their medication changed as a result of the intervention. Oral presentation presented at the 2nd Clinical Trials Methodology Conference 2013: Methodology matters, 18-19 November 2013, Edinburgh, UK

Feasibility of the MELD score as a screening tool for pharmacists to identify patients with impaired hepatic function at hospital admission

WHAT IS KNOWN AND OBJECTIVE Hepatic impairment (HI) is a known risk factor for drug safety. The MELD score (Model-for-endstage-liver-disease), calculated from serum creatinine, bilirubin and International Normalized Ratio (INR), is a promising screening tool corresponding to Child-Pugh Score (CPS) for drug adjustment. We tested the feasibility of MELD as an automatic screening tool accounting for correct calculation, interfering factors (IF) and detection of patients corresponding to CPS-B/C potentially requiring drug adjustment. METHODS We retrospectively calculated MELD for a 3-month cohort of surgical patients and assessed need for adjustment of MELD parameters to standard values. IF for INR (oral anticoagulants) and serum creatinine (renal insufficiency (RI; eGFR\textless60~ml/min/1.73m²); as well as drugs elevating creatinine levels (DECL)) and the number of patients with MELD scores corresponding to CPS-B/C were analysed. For MELD \geq7.5, liver and bile diagnoses were recorded. RESULTS AND DISCUSSION Of 1183 patients, MELD was calculable for 761 (64%; median 7.5, range 6.4-36.8). Parameters had to be adjusted for 690 (91%) patients. IF of parameters were RI in 172 (23%), INR-elevating drugs in 105 (14%) and DECL in 33 (4%) patients. Of 335 (44%) patients with MELD \geq7.5, 122 (36%) had documented liver or bile diagnoses. MELD 10-\textless15 (corresponding to CPS-B) was found for 105 (14%), MELD \geq15 (corresponding to CPS-C) for 66 (9%) of the 761 patients with a calculated MELD. Referred to all patients, drug adjustments due to possible HI were recommendable for 14% of patients with suspected CPS-B/C. WHAT IS NEW AND CONCLUSION MELD is a feasible screening tool for HI as a risk factor for drug safety at hospital admission when appropriately considering correct parameter adjustment and RI and INR-elevating drugs as IF. Further evaluation of sensitivity and specificity is needed

Hepatic Impairment as a Risk Factor for Drug Safety: Suitability and Comparison of Four Liver Scores as Screening Tools

Hepatic impairment (HI) influences the pharmacokinetics and pharmacodynamics of drugs and represents an important risk factor for drug safety. A reliable screening tool for HI identification at hospital admission by pharmacists would be desirable but is currently lacking. Therefore, we tested four liver scores as potential screening instruments. We retrospectively recorded liver/bile diagnoses, symptoms and abnormalities (summarized as hepatic findings) of 200 surgical patients followed by an assessment of the relevance of these findings for drug therapy (rating). The agreement between the Model of Endstage Liver Disease (MELD), Non-alcoholic fatty liver disease fibrosis score (NFS), Fibrosis 4 index (FIB-4), and aspartate-aminotransferase to platelet ratio index (APRI) and the rating was quantified by Cohen’s Kappa. The performance of the scores in this setting was further evaluated by their sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Of 200 patients, 18 (9%) had hepatic findings relevant for drug therapy. Fair agreement was found for FIB-4 and MELD and slight agreement for APRI and NFS compared to the rating. The highest values for sensitivity, specificity, PPV, and NPV were 41.2% (MELD), 99.3% (APRI), 66.7% (APRI), and 93.6% (MELD), respectively. Due to low performance, none of the scores can be recommended for clinical use as a single screening tool for HI at hospital admission